Go as a Scripting Language

Go's growing adoption as a programming language that can be used to create high-performance networked and concurrent systems has been fueling developer interest in its use as a scripting language. While Go is not currently ready "out of the box" to be used as a replacement for Bash or Python, this can be done with a little effort.

As Codelang's Elton Minetto explained, Go has quite some appeal to be used as a scripting language, including its power and simplicity, support for goroutines, and more. Google software engineer Eyal Posener adds more reasons to adopt Go as a scripting language, such as the availability of a rich set of libraries and the language terseness, which makes maintenance easier. On a slightly different note, Go contributor and former Googler David Crawshaw highlights the convenience of using Go for scripting tasks for all programmers spending most of their time writing more complex programs in Go:

Basically, I write Go all the time. I occasionally write bash, perl, or python. Occasionally enough, that those languages fall out of my head.

Being able to use the same language for day-to-day tasks and less frequent scripting task would greatly improve efficiency. Go is also a strongly typed language, notes Cloudflare engineer Ignat Korchagin, which can help to make Go scripts more reliable and less prone to runtime failure due to such trivial errors as typos.

Codenation used Go to create scripts to automate repetitive tasks, both as part of their development workflow and within their CI/CD pipeline. At Codenation, Go scripts are executed by means of go run, a default tool in Go toolchain that compiles and runs a Go program in one step. Actually, go run is not as interpreter, writes Posener:

[...] bash and python are interpreters - they execute the script while they read it. On the other hand, when you type go run, Go compiles the Go program, and then runs it. The fact that the Go compile time is so short, makes it look like it was interpreted.

To make Go scripts well-behaved citizens among shell scripts, Codenation engineers use a number of useful Go packages, including:

• github.com/fatih/color to colorize Go output.

• github.com/schollz/progressbar to create progress bars for lengthy operations.

• github.com/jimlawless/whereami to capture information on the filename, line number, function, etc. where it is used. This is useful to improve error messages.

• github.com/spf13/cobra to make it easier to create complex scripts with input processing, options, and related documentation.

While using go run to run Go program from the command line works well for Codenation, it is far from a perfect solution, writes Crawshaw. In particular, Go lacks support for a read-eval-print loop (REPL) and cannot be easily integrated with the shebang (#!), which enables the execution of a script as if it were a binary program. Additionally, Go error handling is more appropriate for larger programs than it is for shorter scripts. For all of those reasons, he started working on Neugram, a project aiming to create a Go clone solving all of the above limitations. Sadly, Neugram appears now abandoned, possibly due to the complexity of replicating all the fine bits of Go syntax.

A similar approach to Neugram is taken by gomacro, a Go interpreter that also supports Lisp-like macros as a way to both generate code as well as implement some form of generics.

gomacro is an almost complete Go interpreter, implemented in pure Go. It offers both an interactive REPL and a scripting mode, and does not require a Go toolchain at runtime (except in one very specific case: import of a 3rd party package at runtime).

Besides being well suited for scripting, gomacro also aims to enable to use Go as an intermediate language to express detailed specification to be translated into standard Go, as well as to provide a Go source code debugger.

While gomacro provides the most flexibility to use Go for scripting, it is unfortunately not standard Go, which raises another set of concerns. Posener carries through a detailed analysis of the possibilities to use standard Go as a scripting language, including a workaround for the missing shebang. However, each approach falls short in some way or another.

As it seems, there is no perfect solution, and I don’t see why we shouldn’t have one. It seems like the easiest, and least problematic way to run Go scripts is by using the go run command. [...] This is why I think there is still work do be done in this area of the language. I don’t see any harm in changing the language to ignore the shebang line.

For Linux systems, though, there might be an advanced trick which makes it possible to run Go scripts from the command line with full shebang support. This approach, illustrated by Korchagin, relies on shebang support being part of the Linux kernel and on the possibility to extend supported binary formats from the Linux userspace. To make a long story short, Korchagin suggests to register a new binary format in the following way:

$ echo ':golang:E::go::/usr/local/bin/gorun:OC' | sudo tee /proc/sys/fs/binfmt_misc/register
:golang:E::go::/usr/local/bin/gorun:OC

This makes it possible to set the executable bit of a fully standard .go program such as:

package main

import (
    "fmt"
    "os"
)

func main() {
    s := "world"

    if len(os.Args) > 1 {
        s = os.Args[1]
    }

    fmt.Printf("Hello, %v!", s)
    fmt.Println("")

    if s == "fail" {
        os.Exit(30)
    }
}

And execute it with:

$ chmod u+x helloscript.go
$ ./helloscript.go
Hello, world!
$ ./helloscript.go gopher
Hello, gopher!
$ ./helloscript.go fail
Hello, fail!
$ echo $?
30

While this approach will not give provide a REPL, the shebang commodity could be enough for typical use cases. Korchagin's article is full of insights and detailed information about how binary formats work on the Linux kernel, and so interested readers are strongly recommended to read this.